Multifunction coolant manifold structures

ABSTRACT

A cooling system is provided which includes, for instance, a coolant supply manifold, a multifunction coolant manifold structure, and multiple cooling structures. The multifunction coolant manifold structure includes a coolant-commoning manifold and an auxiliary coolant reservoir above and in fluid communication with the coolant-commoning manifold. The multiple cooling structures are coupled in parallel fluid communication between the coolant supply and coolant-commoning manifolds to receive coolant from the supply, and exhaust coolant to the coolant-commoning manifold. The coolant-commoning manifold is sized to slow therein a flow rate of coolant exhausting from the multiple cooling structures to allow gas within the exhausting coolant to escape the coolant within the coolant-commoning manifold. The escaping gas rises to the auxiliary coolant reservoir and is replaced within the coolant-commoning manifold by coolant from the auxiliary coolant reservoir.

BACKGROUND

The power dissipation of integrated circuit chips, and the modulescontaining the chips, continues to increase in order to achieveincreases in processor performance. This trend poses cooling challengesat the module, system, rack and data center levels.

In many large server applications, processors along with theirassociated electronics (e.g., memory, disk drives, power supplies, etc.)are packaged in removable drawer configurations stacked within anelectronics rack or frame comprising information technology (IT)equipment. In other cases, the electronics may be in fixed locationswithin the rack or frame. Conventionally, the components have beencooled by air moving in parallel airflow paths, usually front-to-back,impelled by one or more air moving assemblies (e.g., axial orcentrifugal fans). In some cases it has been possible to handleincreased power dissipation within a single drawer or system byproviding greater airflow, for example, through the use of more powerfulair moving assemblies or by increasing the rotational speed (i.e., RPMs)of the fan mechanisms. However, this approach is becoming problematic,particularly in the context of a computer center installation (i.e.,data center).

The sensible heat load carried by the air exiting the rack(s) isstressing the capability of the room air-conditioning to effectivelyhandle the load. This is especially true for large installations with“server farms” or large banks of computer racks located close together.In such installations, liquid-cooling is an attractive technology toselectively manage the higher heat fluxes. The liquid absorbs the heatdissipated by the components/modules in an efficient manner. Typically,the heat is ultimately transferred from the liquid coolant to a heatsink, whether air or other liquid-based.

BRIEF SUMMARY

The shortcomings of the prior art are addressed and additionaladvantages are provided through the provision, in one aspect of a methodwhich includes providing a cooling system, the providing of the coolingsystem including: providing a coolant supply manifold; providing amultifunction coolant manifold structure which includes acoolant-commoning manifold and an auxiliary coolant reservoir disposedabove and in fluid communication with the coolant-commoning manifold;providing multiple cooling structures coupled in parallel fluidcommunication between the coolant supply manifold and thecoolant-commoning manifold to receive coolant from the coolant supplymanifold, and to exhaust coolant to the coolant-commoning manifold; andwherein the coolant-commoning manifold is sized to slow therein a flowrate of coolant exhausting from the multiple cooling structures to allowgas within the exhausting coolant to escape the coolant within thecoolant-commoning manifold, the escaping gas rising from thecoolant-commoning manifold to the auxiliary coolant reservoir, and beingreplaced within the coolant-commoning manifold by coolant from theauxiliary coolant reservoir.

Additional features and advantages are realized through the techniquesof the present invention. Other embodiments and aspects of the inventionare described in detail herein and are considered a part of the claimedinvention.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

One or more aspects of the present invention are particularly pointedout and distinctly claimed as examples in the claims at the conclusionof the specification. The foregoing and other objects, features, andadvantages of the invention are apparent from the following detaileddescription taken in conjunction with the accompanying drawings inwhich:

FIG. 1 depicts one embodiment of a raised floor layout of an air-cooleddata center, which may employ one or more cooling systems, in accordancewith one or more aspects of the present invention;

FIG. 2 is a cross-sectional elevational view of one implementation of anelectronics rack for a data center, which may employ a cooling system,in accordance with one or more aspects of the present invention;

FIG. 3A is a schematic of one embodiment of a cooled electronicsassembly having multiple electronic systems and a cooling system, inaccordance with one or more aspects of the present invention;

FIG. 3B is a plan view of one embodiment of an electronic system layoutillustrating, by way of example, a hybrid cooling approach for coolingcomponents of the electronic system using, in part, a cooling system, inaccordance with one or more aspects of the present invention;

FIG. 4 is a schematic of another embodiment of a cooled electronicsassembly having multiple electronic systems and a cooling system, inaccordance with one or more aspects of the present invention;

FIG. 5A depicts one detailed embodiment of a partially-assembled, cooledelectronic assembly comprising multiple electronic systems and a coolingsystem, in accordance with one or more aspects of the present invention;

FIG. 5B is an enlarged depiction of one embodiment of the multifunctioncoolant manifold structure of the cooling system of FIG. 5A, inaccordance with one or more aspects of the present invention;

FIG. 6A depicts another embodiment of a partially-assembled, cooledelectronic assembly comprising multiple electronic systems and a coolingsystem, in accordance with one or more aspects of the present invention;and

FIG. 6B is an enlarged depiction of one embodiment of the multifunctioncoolant manifold structure of the cooling system of FIG. 6A, inaccordance with one or more aspects of the present invention.

DETAILED DESCRIPTION

As used herein, the terms “electronics rack” and “rack unit” are usedinterchangeably, and unless otherwise specified include any housing,frame, rack, compartment, blade server system, etc., having one or moreheat-generating components of a computer system, electronic system,information technology (IT) equipment, etc., and may include, forexample, a stand-alone computer processor having high-, mid- or low-endprocessing capability. In one embodiment, an electronics rack maycomprise a portion of an electronic system, a single electronic system,or multiple electronic systems, for example, in one or more subsystems,sub-housings, blades, drawers, nodes, compartments, boards, etc., havingone or more heat-generating electronic components disposed therein orthereon. An electronic system may be movable or fixed, for example,relative to an electronics rack, with rack-mounted electronic drawers ofa rack unit and blades of a blade center system being two examples ofelectronic systems of an electronics rack to be cooled. In oneembodiment, an electronic system may comprise multiple electroniccomponents, and may be, in one example, a server unit.

“Electronic component” refers to any heat generating electroniccomponent of, for example, an electronic system requiring cooling. Byway of example, an electronic component may comprise one or moreintegrated circuit die and/or other electronic devices to be cooled,including one or more processor die, memory die or memory support die.As a further example, an electronic component may comprise one or morebare die or one or more packaged die disposed on a common carrier.Further, unless otherwise specified herein, the term “coolant-cooledcold plate” refers to any conventional thermally conductive, heattransfer structure having a plurality of channels or passageways formedtherein for flowing of coolant, such as liquid coolant, therethrough.

As used herein, “coolant-to-air heat exchanger” means any heat exchangemechanism characterized as described herein, across which air passes andthrough which coolant, such as liquid coolant, can circulate; andincludes, one or more discrete heat exchangers, coupled either in seriesor in parallel. A coolant-to-air heat exchanger may comprise, forexample, one or more coolant flow paths, formed of thermally conductivetubing (such as copper or other tubing) thermally coupled to a pluralityof fins across which air passes. Size, configuration and construction ofthe coolant-to-air heat exchanger can vary without departing from thescope of the invention disclosed herein. Further, “data center” refersto a computer installation containing one or more electronics racks tobe cooled. As a specific example, a data center may include one or morerows of rack-mounted computing units, such as server units.

One example of the coolant is water. However, the concepts disclosedherein are readily adapted to use with other types of coolant. Forexample, the coolant may comprise a brine, a dielectric liquid, afluorocarbon liquid, a liquid metal, or other coolant, or refrigerant,while still maintaining the advantages and unique features of thepresent invention.

Reference is made below to the drawings, wherein the same or similarreference numbers used throughout different figures designate the sameor similar components.

As shown in FIG. 1, in one implementation of a raised floor layout of anair-cooled data center 100, multiple electronics racks 110 are disposedin one or more rows. A computer installation such as depicted in FIG. 1may house several hundred, or even several thousand microprocessors. Inthe arrangement of FIG. 1, chilled air enters the computer room viafloor vents from a supply air plenum 145 defined between a raised floor140 and a base or sub-floor 165 of the room. Cooled air is taken inthrough louvered covers at air inlet sides 121 of the electronics racksand expelled through the back (i.e., air outlet sides 131) of theelectronics racks. Electronics racks 110 may have one or more air-movingdevices (e.g., axial or centrifugal fans) to provide forcedinlet-to-outlet air flow to cool the electronic components within therack units. The supply air plenum 145 provides (in one embodiment)conditioned and cooled air to the air-inlet sides of the electronicsracks via perforated floor tiles 160 disposed in a “cold” aisle of thecomputer installation. The cooled air is supplied to plenum 145 by oneor more air conditioning units 150, also disposed within data center100. Room air is taken into each air conditioning unit 150 near an upperportion thereof. This room air may comprise (in part) exhausted air fromthe “hot” aisles of the computer installation defined by opposing airoutlet sides 131 of the electronics racks 110.

FIG. 2 depicts (by way of example) one embodiment of an electronics rack110 with a plurality of electronic systems 201 to be cooled. In theembodiment illustrated, electronic systems 201 are air-cooled by coolairflow 202 ingressing via air inlet side 121, and exhausting out airoutlet side 131 as heated airflow 203. By way of example, one or moreair-moving assemblies 208 may be provided at the air inlet sides ofelectronic systems 201 and/or one or more air-moving assemblies 209 maybe provided at the air outlet sides of electronic systems 201 tofacilitate airflow through the individual systems 201 as part of thecooling apparatus of electronics rack 110. For instance, air-movingassemblies 208 at the air inlets to electronic systems 201 may be orinclude axial fan assemblies, while air-moving assemblies 209 disposedat the air outlets of electronic systems 201 may be or includecentrifugal fan assemblies. One or more electronic systems 201 mayinclude heat-generating components to be cooled of, for instance, anelectronic subsystem, and/or information technology (IT) equipment. Moreparticularly, one or more of the electronic systems 201 may include oneor more processors and associated memory.

In one embodiment, electronics rack 110 may also include, by way ofexample, one or more bulk power assemblies 204 of an AC to DC powersupply assembly. AC to DC power supply assembly further includes, in oneembodiment, a frame controller, which may be resident in the bulk powerassembly 204 and/or in one or more electronic systems 201. Alsoillustrated in FIG. 2 is one or more input/output (I/O) drawer(s) 205,which may also include a switch network. I/O drawer(s) 205 may include,as one example, PCI slots and disk drivers for the electronics rack.

In the depicted implementation, a three-phase AC source feeds power viaan AC power supply line cord 206 to bulk power assembly 204, whichtransforms the supplied AC power to an appropriate DC power level foroutput via distribution cable 207 to the plurality of electronic systems201 and I/O drawer(s) 205. The number of electronic systems installed inthe electronics rack is variable, and depends on customer requirementsfor a particular system. Note that the particular electronics rack 110configuration of FIG. 2 is presented by way of example only, and not byway of limitation. In particular, FIGS. 3A-6B depict, in part, otheralternate implementations of an electronics rack and cooling approaches.

Referring first to FIG. 3A, a schematic diagram is presented of oneembodiment of a cooled electronic assembly configured as a cooledelectronics rack 110′, which includes multiple electronic systems 301and a cooling system, which may be disposed fully or partially internalto the electronics rack, or in an alternate implementation, external,and even remote from the electronics rack. In the depictedimplementation, electronic systems 301 each have an associated coolingstructure (or heat removal structure) of the cooling system. By way ofexample, one or more of the cooling structures may comprise one or morecoolant-cooled cold plates, or one or more coolant-immersion housingsdepending, for instance, whether indirect or direct liquid-assistedcooling is desired. The cooling system further includes a coolant supplymanifold 310 and a coolant-commoning manifold 320, with the multiplecooling structures being coupled in parallel fluid communication betweencoolant supply manifold 310 and coolant-commoning manifold 320 toreceive coolant from the coolant supply manifold, and exhaust coolant tothe coolant-commoning manifold.

FIG. 3A depicts an example of a closed-loop cooling system, withmultiple control and monitor components that allow the system to operatereliably. These components include one or more de-aerators 321 to removedissolved gasses from the coolant, a coolant expansion structure 322 toaccommodate expansion of coolant within the cooling system, a reservoir323, one or more level sensors 324 associated with reservoir 323 tosense level of coolant within the cooling system, a vacuum breaker 325coupled to the coolant loop of the cooling system to prevent cavitationof the pumping assembly, and a pressure-relieve valve 326 associatedwith the coolant loop to ensure that the cooling system does notover-pressurize. A fill port 328 may be provided at the top of thecooling system, and a drain port 329 may be provided in a lower portionof the cooling system. As shown, reservoir 323 functions to supplycoolant to a distribution manifold 330 of a pumping assembly, whichincludes multiple pumping units 335, pump 1, pump 2, pump 3, each withan associated check valve 336. Further, the pumping assembly includes areturn manifold 340. As illustrated, the pumping units 335 of thepumping assembly are coupled in parallel fluid communication betweendistribution manifold 330 and return manifold 340. In oneimplementation, the pumping units 335 are modular pumping units (MPUs),which may be individually, selectively replaced concurrent withcontinued operation of the cooling system of the cooled electronicassembly depicted. Note that, in one implementation, the components ofthe cooling system of FIG. 3A are discrete components which fulfill theabove-described functions.

As illustrated, the cooling system further includes a heat removalsection 350, coupled in fluid communication between return manifold 340of the pump assembly and coolant supply manifold 310. By way of example,heat removal section 350 includes one or more coolant-to-air heatexchangers with one or more associated fan mechanisms (e.g., axial orcentrifugal fans) to facilitate air-cooling of coolant within the heatexchanger(s) by flowing cooled air 300 across heat removal section 350.After passing across heat removal section 350, the heated air egressesfrom the rack unit as heated air 300′. Note that in an alternateembodiment, the heat removal section could include one or morecoolant-to-coolant heat exchangers, or one or more liquid-to-liquid heatexchangers, to reject heat from the coolant circulating through thecooling system. For instance, the heat could be rejected tofacility-chilled water where available, rather than to cooled air 300.

In operation, heat generated within the electronic systems 301 isextracted by coolant flowing through (for example) respective coolingstructures associated therewith, such as cold plates, and is returnedvia the coolant-commoning manifold 320 and the active modular pumpingunit(s) (MPU) 335, for example, for rejection of the heat from thecoolant to the cooled ambient air 300 passing across the heat exchangerin heat removal section 350. In one implementation, only one modularpumping unit 335 may (depending on the mode) be active at a time, andthe MPU redundancy allows for, for example, servicing or replacement ofan inactive modular pumping unit from the cooling system, withoutrequiring shut-off of the electronic systems being cooled. By way ofspecific example, quick connect couplings may be employed, along withappropriately sized and configured hoses to couple, for example, theheat exchanger, cold plates, supply and return manifolds, reservoir andpumping units. Redundant fan mechanisms, such as redundant centrifugalfans, with appropriate, redundant drive cards or controllers, may bemounted to direct cooled air 300 across the heat exchanger(s) of theheat removal section. These controllers may be in communication with asystem-level controller (not shown), in one embodiment. In one normalmode implementation, the multiple fan mechanisms may be running at thesame time.

Auxiliary (or backup) air-cooling may be provided across the electronicsystems 301, for instance, in the case of a failure of the coolant-basedcooling apparatus which requires shut-off of coolant flow to theelectronic systems 301. In such a case, airflow may be drawn through therack from an air inlet side to an air outlet side thereof via redundantbackup fan mechanisms (not shown) and appropriate airflow ducting. Notein this regard, that in one embodiment, the auxiliary airflow coolingapparatus may be disposed above the multiple electronic systems withinthe electronics rack, and the coolant-based cooling system discussedherein may be disposed below the multiple electronic systems to becooled, as in the schematic of FIG. 3A.

Note that, although depicted with reference to FIG. 3A with respect toone or more coolant-to-air heat exchangers, the cooling system(s)disclosed herein may provide pumped coolant (such as water) forcirculation through various types of heat exchange assemblies, includingone or more coolant-to-air heat exchangers, one or morecoolant-to-coolant heat exchangers, a rack-mounted door heat exchanger,a coolant-to-refrigerant heat exchanger, etc. Further, the heat exchangeassembly may comprise more than one heat exchanger, including more thanone type of heat exchanger, depending upon the implementation. The heatexchange assembly, or more generally, heat removal section, could bewithin the cooled electronics rack, or positioned remotely from therack.

By way of example only, FIG. 3B depicts one embodiment of an electronicsystem 301 component layout wherein one or more air moving devices 208provide forced air flow 300 to cool multiple components 304 withinelectronic system 301. Cool air is taken in through a front 302 andexhausted out a back 303 of the drawer. The multiple components to becooled include multiple processor modules to which cooling structures,such as coolant-cooled cold plates 305 of the cooling system arecoupled, as well as multiple arrays of memory modules 306 (e.g., dualin-line memory modules (DIMMs)) and multiple rows of memory supportmodules 307 (e.g., DIMM control modules) to which air-cooled heat sinksmay be coupled. In the embodiment illustrated, memory modules 306 andmemory support modules 307 are partially arrayed near front 302 ofelectronic system 301, and partially arrayed near back 303 of electronicsubsystem 301. Also, in the embodiment of FIG. 3B, memory modules 306and the memory support modules 307 are cooled by air flow 300 across theelectronic system.

The illustrated coolant-based cooling system further includes multiplecoolant-carrying tubes connected to and in fluid communication withcoolant-cooled cold plates 305. The coolant-carrying tubes comprise setsof coolant-carrying tubes, with each set including (for example) acoolant supply tube 315, a bridge tube 316 and a coolant return tube317. In this example, each set of tubes provides coolant to aseries-connected pair of cold plates 305 (coupled to a pair of processormodules). Coolant flows into a first cold plate of each pair via thecoolant supply tube 315 and from the first cold plate to a second coldplate of the pair via bridge tube or line 316, which may or may not bethermally conductive. From the second cold plate of the pair, coolant isreturned through the respective coolant return tube 317.

In FIGS. 3A & 3B, a closed-loop cooling system is illustrated whichincorporates a number of components that ensure that the cooling systemworks reliably. These include, but are not necessarily limited to: acoolant reservoir; coolant level sensors; a coolant expansion region;one or more vacuum breakers; one or more pressure-relieve valves; apumping assembly which may include multiple modular pumping units;distribution and return manifolds for pump flow through one or moreparallel-coupled pumping units of the pump assembly; check valves toprevent back flow through one or more inactive pumps of the pumpassembly; a separate de-aerator facility to remove air or other gassesfrom the coolant within the cooling system; a supply manifold todistribute coolant to multiple cooling structures coupled in parallel; acoolant supply manifold; a return manifold to receive exhaust coolantfrom the multiple cooling structures; a heat removal section ormechanism, such as a coolant-to-air heat exchanger; and fill and drainports for filling and draining the cooling system.

In one implementation, the above-noted components of the cooledelectronic assembly, and in particular, the noted components of thecooling system, may be discrete components obtained, at least in part,as commercially available components. However, implementing the coolingsystem in this manner may add cost, space, and complexity to the coolingsystem, as well as to the resultant cooled electronic assembly. Inaccordance with aspects of the present invention, many of theabove-noted structures or functions may be integrated (or combined)within a single, novel, multifunction coolant manifold structure.

For instance, in one embodiment, the multifunction coolant manifoldstructure may include or provide: a coolant reservoir; one or morecoolant level sensors; a coolant expansion region; one or more vacuumbreakers to prevent pump cavitation; one or more pressure-relief valvesto ensure the cooling system does not over-pressurize; a distributionmanifold to distribute coolant to the pumping assembly; a de-aeratorfacility to remove air and other gasses from the coolant within thecooling system; a coolant-commoning manifold to common exhaust coolantfrom multiple cooling structures; as well as one or more fill or drainports for the cooling system. Advantageously, combining components ofthe cooling system into a single, multipurpose manifold structure savescost, reduces space, and reduces complexity of the cooling system, aswell as of the resultant cooled electronic assembly.

Generally stated, disclosed herein are cooling systems, cooledelectronic assemblies, and methods of fabrication, which include amultifunction coolant manifold structure. For instance, the coolingsystem may include a coolant supply manifold, a multifunction coolantmanifold structure, and multiple cooling structures. The multifunctioncoolant manifold structure includes, in one embodiment, acoolant-commoning manifold and an auxiliary coolant reservoir, which maybe disposed above and in fluid communication with the coolant-commoningmanifold. The multiple cooling structures are coupled in parallel fluidcommunication between the coolant supply manifold and thecoolant-commoning manifold to receive coolant from the coolant supplymanifold, and exhaust coolant to the coolant-commoning manifold. Thecoolant-commoning manifold is sized to slow a flow rate of coolantexhausting from the multiple cooling structures to allow gas within theexhausting coolant to escape the coolant within the coolant-commoningmanifold. The multifunction coolant manifold structure is configured forthe escaping gas (e.g., air bubbles) to rise to the auxiliary coolantreservoir, and be replaced within the coolant-commoning manifold bycoolant from the auxiliary coolant reservoir.

In certain implementations, the multifunction coolant manifold structureis a single, integrated and rigid structure, where the auxiliary coolantreservoir is integrated with the coolant-commoning manifold. In thisconfiguration, the coolant-commoning manifold may have a largerdimension in a first direction, such as the vertical direction, comparedwith that of the auxiliary coolant reservoir, which may have a largerdimension in a second direction, such as the horizontal direction. Thus,in one embodiment, the coolant-commoning manifold may be an elongate,vertical manifold, and the auxiliary coolant reservoir may have a largercross-sectional area in a horizontal direction to accommodate additionalcoolant.

In certain implementations, the auxiliary coolant reservoir is coupledin fluid communication with the coolant-commoning manifold via adetachable coolant conduit or hose. In this configuration, thecoolant-commoning manifold may be the same size as, or have a largervolume than, the auxiliary coolant reservoir. Alternatively, in one ormore implementations, the auxiliary coolant reservoir may have a largervolume of coolant than the coolant-commoning manifold. Also note that,in one or more embodiments, the coolant-commoning manifold may have acoolant volume twice or larger the size of the coolant volume of thecoolant supply manifold of the cooling system.

In certain embodiments, the multifunction coolant manifold structureincludes a detachable, field-replaceable unit, which comprises theauxiliary coolant reservoir. Further, the field-replaceable unit mayinclude one or more components for at least one of monitoring orcontrolling one or more characteristics of coolant within themultifunction coolant manifold structure. By way of example, the one ormore components may include one or more coolant level sensors (forsensing a level of coolant within the manifold structure); one or morevacuum breakers (to prevent cavitation within the pumping assembly ofthe cooling system); and/or one or more pressure-relief valves (toensure that the cooling system does not over-pressurize), etc.Advantageously, by associating these components with thefield-replaceable unit, the one or more components may be readilyremoved for servicing or replacement by simply exchanging out thefield-replaceable unit of the multifunction coolant manifold structure.Further, by sizing the coolant-commoning manifold as discussed herein,and by locating the field-replaceable unit above the coolant-commoningmanifold, the field-replaceable unit may be replaced while the coolingsystem is operational, that is, while coolant continues to be pumpedthrough the cooling system to cool the electronic systems. This can beaccomplished, in part, by utilizing quick disconnect couplings inassociation with the detachable coolant conduit coupling the auxiliarycoolant reservoir to the coolant-commoning manifold.

In one or more embodiments, a pumping assembly is provided to circulatecoolant through the cooling system, where the pumping assembly iscoupled in fluid communication to the multifunction coolant manifoldstructure via one or more coolant distribution connections. In thisimplementation, the multifunction coolant manifold structure includesboth the coolant-commoning manifold and a coolant distribution manifoldportion in a single, rigid manifold structure. In certain embodiments,the pumping assembly includes multiple coolant pumps (such as two ormore modular pumping units (MPUs)), coupled in parallel fluidcommunication to the multifunction coolant manifold structure forselectively pumping coolant in parallel from the multifunction coolantmanifold structure. The multiple coolant pumps facilitate continuedoperation of the cooling system. In operation, only one pumping unit may(depending on the mode) be active at a time, with modular pumping unit(MPU) redundancy allowing for, for example, servicing or replacement ofan inactive modular pumping unit from the cooling system, withoutrequiring shut-off of the electronic systems or electronics rack beingcooled. By way of specific example, quick connect couplings may beemployed, along with appropriately sized and configured hoses to couple,for example, the multifunction coolant manifold structure, pumpingassembly, heat removal section, and coolant supply manifold, as well asthe multiple cooling structures associated with the electronic systemsto be cooled.

In certain embodiments, the auxiliary coolant reservoir incorporates acoolant expansion region in an upper portion thereof, and a coolant fillport disposed below a top of the auxiliary coolant reservoir, whichassists in defining an air pocket in the upper portion of the reservoiras the coolant expansion region. In one or more implementations, theauxiliary coolant reservoir further includes one or more vacuum breakersto prevent cavitation within the cooling system, and/or one or morepressure-relief valves to prevent the cooling system fromover-pressurizing. Further, in certain implementations, the auxiliarycoolant reservoir may have one or more coolant level sensors associatedtherewith to sense a level of coolant within the reservoir, or moregenerally, within the multifunction coolant manifold structure.

FIG. 4 is a schematic depiction of one embodiment of a cooled electronicassembly configured as a cooled electronics rack 110″, similar to theabove-described cooled electronics rack 110′ of FIGS. 3A & 3B. Onesignificant difference in the assembly configuration of FIG. 4, however,is the provision of a multifunction coolant manifold structure 400,which integrates many of the functions and components described above inconnection with the cooling system provided for the cooled electronicassembly of FIGS. 3A & 3B. In particular, as illustrated in FIG. 4, themultifunction coolant manifold structure 400 is shown to include, in oneembodiment, a coolant-commoning manifold 320′, a de-aerator facility321′, a coolant expansion region 322′, a coolant reservoir 323′, one ormore coolant level sensors 324′, one or more vacuum breakers 325′, oneor more pressure-relieve valves 326′, a distribution manifold 330′ forthe pump assembly, as well as one or more fill or drain ports 328′,329′. As described further below in connection with the embodiments ofFIGS. 5A-6B, these components are differently configured, however,and/or alternately implemented in comparison to the discrete componentsemployed in the cooling system described above in connection with FIGS.3A & 3B.

By way of example, FIGS. 5A & 5B depict one detailed embodiment of apartial cooled electronic assembly, in accordance with one or moreaspects of the present invention. In the depicted embodiment, the cooledelectronic assembly includes a cooling system housing 500, which may beconfigured for disposition within, for instance, a lower portion of anelectronics rack, such as within one or more of the above-describedelectronics racks. As illustrated, the cooled electronic assembly alsoincludes multiple electronic systems 301 to be cooled. In theconfiguration of FIG. 5A, the electronic systems are shown (by way ofexample only) one above the other, above cooling system housing 500, asthey might be positioned within an electronics rack. Note, however, thatthis particular configuration is presented as one example only.Electronic systems 301 each have associated therewith a coolingstructure (not shown), such as a coolant-cooled heat sink, cold plate,immersion-cooling housing, etc., which facilitates extraction of heatfrom the respective electronic system, or from one or more electroniccomponents within the respective electronic system.

As illustrated in FIG. 5A, the cooling system includes, in the depictedembodiment, a coolant supply manifold 310, which includes respectivequick connect couplings 501 that facilitate connection of appropriatelysized and configured hoses 503 to the coolant supply manifold 310, so asto couple in fluid communication the coolant supply manifold and thecooling structures associated with the electronic systems 301. Similarhoses 505 and quick connect couplings 502 are associated with themultifunction coolant manifold structure 400 of the cooling system forcoupling the cooling structures associated with the electronics systems301 in parallel fluid communication with manifold structure 400 as well.

As illustrated in FIG. 5A, the multifunction coolant manifold structureincludes a coolant-commoning manifold 320′ from which quick connectcouplings 502 extend. In one embodiment, coolant-commoning manifold 320′is sized larger than coolant supply manifold 310 (e.g., 2× larger orgreater in coolant volume) to, in part, slow a flow rate of coolantexhausting from the cooling structures associated with the electronicsystems 301 as the coolant enters the coolant-commoning manifold 320′.This slowing of the coolant flow rate is designed so that entrained airor gas within the coolant is allowed to escape within thecoolant-commoning manifold 320′ and rise, in one embodiment, to anauxiliary coolant reservoir 323′ located above the coolant-commoningmanifold 320′, and in fluid communication therewith.

In the example of FIGS. 5A & 5B, the multifunction coolant manifoldstructure 400 is a single, integrated and rigid structure, with thecoolant-commoning manifold 320′ and auxiliary coolant reservoir 323′ influid communication within the integrated structure. As escaping air orgas rises to the auxiliary coolant reservoir 323′ from thecoolant-commoning manifold 320′, it is replaced within thecoolant-commoning manifold 320′ by coolant from the auxiliary coolantreservoir 323′. That is, as air or gas rises, coolant drops from theauxiliary coolant reservoir 323′ into the coolant-commoning manifold320′. In this manner, the multifunction coolant manifold structure 400inherently functions as a de-aerator. Further, a coolant expansionregion is defined in an upper portion of auxiliary coolant reservoir323′ by providing, for instance, a coolant fill port 328′ in associationwith the auxiliary coolant reservoir on a side of the reservoir, spacedbelow an upper-most (or top) of the auxiliary coolant reservoir 323′. Inthis manner, a volume of air (that is, an air pocket) is formed abovethe coolant fill port 328′ within the auxiliary coolant reservoir. Thisvolume of air advantageously allows for safe expansion and contractionof the coolant within the cooling system due, for instance, to changingtemperatures or pressures.

As illustrated in FIGS. 5A & 5B, the multifunction coolant manifoldstructure 400, and in particular, the auxiliary coolant reservoir 323′portion thereof, includes connections for one or more components to atleast one of monitor or control one or more characteristics of thecoolant within the multifunction coolant manifold structure. These oneor more components may include, for instance, one or more coolant levelsensors 324′ for sensing level of coolant within the multifunctioncoolant manifold structure 400, and reporting the level to a coolingsystem controller (not shown) for use in possible control action. Forinstance, should the level of coolant within the multifunction coolantmanifold structure drop to an unacceptably low level, the levelsensor(s) 324′ signals could be employed by the controller to signal aservice operator to add coolant to the system. Alternatively, dependingon the sensed level, the controller could automatically shut the coolingsystem down, and depending on the implementation, possibly shut theelectronic systems down as well. This might depend, for instance, onwhether backup cooling, such as backup airflow cooling, is integratedwithin the cooled electronic assembly. Additionally, the componentconnections may allow for connections of one or more vacuum breakers325′, and/or one or more pressure-relief valves 326′, as describedabove.

As shown in the figures, the multifunction coolant manifold structure400 further includes a coolant distribution manifold portion withcoolant distribution connections 507 (FIG. 5B), which allow coolanthoses 510 (FIG. 5A) to couple to the manifold structure to receivecoolant from the multifunction coolant manifold structure fordistribution to multiple pumping units, such as the above-describedmodular pumping units. In the embodiment of FIG. 5A, three modularpumping units 335′ are illustrated by way of example only, eachreceiving (via the respective hose connections 507) coolant from themultifunction coolant manifold structure. In one implementation, theparallel-coupled pumping units 335′ operate to independently pumpcoolant through a return manifold to a heat removal section (asdescribed above), which may also be disposed within the cooling systemhousing 500, for instance, behind the depicted pumping units 335′. Inone implementation, the heat removal section may comprise one or morecoolant-to-air heat exchangers, with air being drawn through the coolingsystem housing 500 via one or more fan mechanisms, which in oneembodiment, may also be disposed within the housing, for instance,behind the one or more coolant-to-air heat exchangers. In an alternateembodiment, the heat removal section could include one or moreliquid-to-liquid heat exchangers to reject heat from the coolantcirculating through the cooling system to, for instance,facility-chilled liquid, such as building-chilled water. The heatremoval section is coupled to coolant supply manifold 310 via a hose 520and appropriate connections.

As illustrated in FIG. 5A, the cooling system may include, in oneembodiment, a coolant drain hose 521 coupled in fluid communication withthe heat removal section and disposed at a lower-most portion of thecooling system to facilitate selective draining of coolant from thecooling system, or filling of coolant into the cooling system, dependingon the current life stage of the system. An appropriate quick connectcoupling may be provided at the end of drain hose 521 to facilitate theoperation.

Note that FIGS. 5A & 5B depict one embodiment only of an multifunctioncoolant manifold structure 400, configured as an integrated structure,wherein the above-described components or facilities are advantageouslyintegrated into a common, multipurpose structure. By way of example, themultifunction coolant manifold structure may be fabricated of a single,punched, stainless steel sheet metal stamping, which is bent into theappropriate shape and robotically welded to arrive at the desiredstructure. The illustrated manifold structure 400 is used to common theexhaust flow from the parallel-coupled cooling structures associatedwith the electronic systems. In one implementation, these could beparallel computer nodes or server nodes of an electronics rack, withfour electronic systems being illustrated in FIGS. 5A & 5B, by way ofexample only.

As noted, the upper portion of the multifunction coolant manifoldstructure is advantageously configured as an auxiliary coolantreservoir. In one or more implementations, the cross-sectional area ofthe auxiliary coolant reservoir 323′ is larger than the cross-sectionalarea of the coolant-commoning manifold 320′. In particular, in thedepicted implementation, the coolant-commoning manifold 320′ has alarger dimension in a first, vertical direction compared with that ofthe auxiliary coolant reservoir 323′, but that the auxiliary coolantreservoir 323′ has a larger horizontal dimension in a second directioncompared with that of the coolant-commoning manifold 320′. Note that thespecific configuration of auxiliary coolant reservoir 323′ is presentedby way of example only. The size and configuration of the multifunctioncoolant manifold structure may depend, in part, on the available sizewithin the associated electronics assembly or electronics rack to whichthe cooling system provides cooling.

Note that, in one embodiment, the coolant-commoning manifold 320′cross-section is made larger than normally required to carry the coolantflow (for instance, 2× or larger) in order to allow the returning,exhausting coolant to slow down, allowing air and other gas in thecoolant to de-aerate, or come out of solution, within thecoolant-commoning manifold, with any gas bubbles rising to the auxiliarycoolant reservoir portion at the top of the manifold structure, whilecoolant from the reservoir replaces the gas bubbles from thecoolant-commoning manifold. Note that the multifunction coolant manifoldstructure further may incorporate, for example, in association with theauxiliary coolant reservoir, one or more level sensors, to allow thecooling system controller to know the current coolant level state, andtake or signal for action, if required.

Additionally, features or connections may be provided in themultifunction coolant manifold structure, such as, in association withthe auxiliary coolant reservoir (in one embodiment), to facilitateinstalling vacuum breakers 325′ and/or pressure-relief devices 326′. Thevacuum breaker(s) ensures that the auxiliary coolant reservoir is nearatomospheric or slightly negative pressure. This feature may be employedto prevent the pumps from cavitating due to a negative pressure in thesystem. The pressure-relief valves may be provided as a safety feature.These devices and valves are placed, in one embodiment, in the auxiliarycoolant reservoir, at the highest coolant location within the coolingsystem. This ensures that, even if the devices fail in an open state, nocoolant will escape since the coolant is under little or no pressurewithin the multifunction coolant manifold structure. During normaloperation, the devices can fail in place, and not cause any functionalproblems with the cooling system disclosed herein. The component(s) canalso be safely removed while the cooling system is operational. Notethat in the embodiment of FIGS. 5A & 5B, the vacuum breaker devices 325′and pressure-relief valves 326′ are located in the upper-most portion ofthe auxiliary coolant reservoir 323′.

Mounting brackets may be provided to facilitate convenient mounting ofthe coolant supply manifold and multifunction coolant manifold structureinto the electronics rack or frame. Filling and draining of the coolingsystem is facilitated by providing one or more fill or drain ports inassociation with the multifunction coolant manifold structure. In theembodiment of FIG. 5A, a fill or drain port 328′ is provided inassociation with the auxiliary coolant reservoir 323′. This, along withthe discharge hose 521 (FIG. 5A) may be used to fill the cooling system.For instance, the quick connect coupling at the end of drain hose 521may be engaged to pump coolant into the cooling system, and the port328′ associated with the reservoir may be used to vent air during thefilling operation. During draining, the procedure may be reversed, withair being allowed in through port 328′ as coolant drains from drain hose521. In one implementation, the multifunction coolant manifold structureis filled with coolant, as is the rest of the cooling system, prior tostarting the pumping assembly. The reservoir 323′ is, in oneimplementation, sized with a sufficient volume of coolant to ensure thatif an unfilled cooling structure associated with one of the electronicssystems is connected to the cooling system during operation, that therewill be sufficient coolant within the cooling system to continueoperation. Note that in the embodiment presented, a large volume ofcoolant exists above the multiple parallel-coupled pumps, ensuring agood source of coolant to prime the pumping units.

By way of further example, FIGS. 6A & 6B depict an alternate embodimentof a partial cooled electronic assembly, in accordance with one or moreaspects of the present invention. In this embodiment, the cooledelectronic assembly includes cooling system housing 500 configured anddisposed as described above in connection with FIGS. 5A & 5B.

Electronic systems 301 each have associated therewith a coolingstructure, such as a coolant-cooled heat sink, cold plate,immersion-cooling housing, etc., which facilitates extraction of heatfrom the respective electronic system, or from one or more electroniccomponents within the electronic system, to coolant flowing through thecooling structure. The cooling structures associated with electronicsystems 301 are coupled in parallel between coolant supply manifold 310and multifunction coolant manifold structure 400′. In particular,respective quick connect couplings 501 facilitate connection ofappropriately sized and configured hoses 503 to coolant supply manifold310, so as to couple in fluid communication the coolant supply manifoldand the cooling structures associated with the electronic systems.Similar hoses 505 and quick connect couplings 502 are associated withmultifunction coolant manifold structure 400′ of the cooling system forcoupling the cooling structures associated with the electronic systems301 in parallel fluid communication with coolant-commoning manifold320′, as illustrated.

As in the embodiment of FIGS. 5A & 5B, coolant-commoning manifold 320′is sized larger (for example, 2× or greater coolant volume) than coolantsupply manifold 310 to, in part, slow a flow rate of coolant exhaustingfrom the cooling structures associated with electronic systems 301 asthe coolant enters the coolant-commoning manifold 320′. This slowing ofthe coolant flow rate is configured or designed so that entrained air orgas within the coolant is allowed to escape the coolant within thecoolant-commoning manifold 320′, and rise, in one embodiment, to adetachable, field-replaceable unit 600 comprising auxiliary coolantreservoir 323′. In the embodiment illustrated, field-replaceable unit600 is located above the coolant-commoning manifold 320′, and in fluidcommunication therewith via, for instance, one or more hose connections601 and an appropriately sized and configured detachable coolant conduit602 coupling, for instance, an upper portion of coolant-commoningmanifold 320′ to a lower portion of auxiliary coolant reservoir 323′, asillustrated in FIGS. 6A & 6B.

In the example of FIGS. 6A & 6B, the multifunction coolant manifoldstructure 400′ comprises, in part, coolant-commoning manifold 320′ andthe separate field-replaceable unit 600, which includes the auxiliarycoolant reservoir 323′. The two structures are in fluid communicationvia detachable coolant conduit 602. Thus, as escaping air or gas risesto the auxiliary coolant reservoir 323′ from the coolant-commoningmanifold 320′, it is replaced within the coolant-commoning manifold 320′by coolant from the auxiliary coolant reservoir 323′. That is, as air orgas rises, coolant drops from the auxiliary coolant reservoir 323′ inthe field-replaceable unit 600 into the coolant-commoning manifold 320′.Thus, the multifunction coolant manifold structure 400′ is sized andconfigured to inherently function as a de-aerator. Further, a coolantexpansion region is defined in an upper portion of auxiliary coolantreservoir 323′ by providing, for instance, coolant fill port 328′ inassociation with the auxiliary coolant reservoir 323′ on a side of thereservoir, spaced below an upper-most (or top) of the auxiliary coolantreservoir 323′. In this manner, a volume of air (that is, an air pocket)is formed above the coolant fill port 328′ within the auxiliary coolantreservoir. This volume of air allows for safe expansion and contractionof the coolant within the multifunction coolant manifold structure, andmore generally, within the cooling system, due, for instance, tochanging temperatures or pressures.

As illustrated in FIGS. 6A & 6B, multifunction coolant manifoldstructure 400′, and in particular, field-replaceable unit 600 thereof,includes connections for one or more components to monitor or controlone or more characteristics of the coolant within the multifunctioncoolant manifold structure. These one or more components may include,for instance, one or more coolant level sensors 324′ for sensing levelof coolant within the multifunction coolant manifold structure 400′, andreporting the level to a cooling system controller (not shown) for usein possible control action, as described above in connection with FIGS.5A & 5B. Additionally, the component connections may allow forconnections of one or more vacuum breakers 325′, and/or one or morepressure-relief valves 326′, as described above. In an alternate, orfurther implementation, a pressure-relief valve 611 may be provided onthe end of a conduit 610, which extends from coolant-commoning manifold320′ along the side of field-replaceable unit 600, but not in fluidcommunication therewith. If an excessive pressure event occurs, coolantmay pass up conduit 610, and through pressure-relief valve 611, into asecond conduit 612, which connects from the pressure-relief valve anddirects the exhausting coolant to the bottom of the cooled electronicassembly, such as to the bottom of the electronics rack. Thus, anycoolant released during the pressure event will safely discharge throughthe conduits 610 & 612, in one implementation.

Advantageously, by associating one or more monitoring or controlcomponents 324′, 325′, 326′ with the field-replaceable unit, thecomponents may be readily removed from the multifunction coolantmanifold structure by simply replacing the field-replaceable unit 600coupled via respective quick connect couplings and conduit 602 tocoolant-commoning manifold 320′. Further, by coupling thefield-replaceable unit above the coolant-commoning manifold, thefield-replaceable unit may be replaced while the cooling system isoperational, with coolant exhausting from the multiple coolingstructures to the coolant-commoning manifold 320′ of the multifunctioncoolant manifold structure 400′.

As in the embodiment of FIGS. 5A & 5B, the multifunction coolantmanifold structure 400′ further includes a coolant distribution manifoldportion with coolant distribution connections 507 (FIG. 6B), which allowcoolant hoses 510 (FIG. 6A) to couple to the manifold structure toreceive coolant therefrom for distribution to multiple pumping units335′, such as described above. Note that in the embodiment of FIG. 6A,three modular pumping units 335′ are illustrated by way of example only,with each receiving, via respective hose connection 507, coolant fromthe multifunction coolant manifold structure. In one implementation, theparallel-coupled pumping units 335′ operate to independently pumpcoolant through a return manifold (not shown) to a heat removal section(as described above), which may also be disposed within cooling systemhousing 500, for instance, behind the depicted pumping units 335′. Inone implementation, the heat removal section includes one or morecoolant-to-air heat exchangers, with air being drawn through the coolingsystem housing 500 via one or more fan mechanisms, which in oneembodiment, may also be disposed within the housing, for instance,behind the one or more coolant-to-air heat exchangers. In an alternateembodiment, the heat removal section could include one or moreliquid-to-liquid heat exchangers to reject heat from the coolantcirculating through the cooling system to, for instance,facility-chilled liquid, such as building-chilled water. The heatremoval section may be coupled to coolant supply manifold 310 via hose520 and appropriate connections.

As with the embodiment of FIGS. 5A & 5B, the cooling system of FIGS. 6A& 6B may include coolant drain hose 521 coupled in fluid communicationwith the heat removal section and disposed at a lower-most portion ofthe cooling system to facilitate selective draining of coolant from thecooling system, or filling of coolant into the cooling system, dependingon the current life stage of the system. Appropriate quick connectcouplings may be provided in association with drain hose 521 tofacilitate the operation.

By way of example, the coolant-commoning manifold structure 320′ and thefield-replaceable unit 600 may each be fabricated of a single, punched,stainless steel sheet metal stamping, which is bent into the appropriateshape and robotically welded to arrive at the desired structure. Theillustrated manifold structure 400′, and in particular, thecoolant-commoning manifold 320′, is used to common the exhaust flow fromthe parallel-coupled cooling structures associated with the electronicsystems. In one implementation, these could be parallel computer nodesor server nodes of an electronics rack, with four electronic systems ofFIG. 6A being illustrated, by way of example only.

In certain applications, the height of the coolant electronic assembly,or more particularly, the height of the electronics rack, may be tootall for a particular customer facility, and in particular, too tall fora particular customer door opening. Thus, a height reduction to theelectronics rack may be required, and in such a case, the multifunctioncoolant manifold structure of FIGS. 6A & 6B may be advantageouslyemployed, where a detachable conduit and quick connect couplings providethe fluid connection between the coolant-commoning manifold and theauxiliary coolant reservoir of the multifunction coolant manifoldstructure.

In one or more implementations, the cross-sectional area of theauxiliary coolant reservoir 323′ is significantly larger than thecross-sectional area of the coolant-commoning manifold 320; that is,taken transversely through the respective structures. In the depictedimplementation, coolant-commoning manifold 320′ has a larger verticaldimension compared with that of the auxiliary coolant reservoir 323′,but the auxiliary coolant reservoir 323′ has, by way of example, alarger horizontal dimension in a second direction compared with that ofthe coolant-commoning manifold 320′. Note that the specificconfiguration of the auxiliary coolant reservoir 323′ of FIGS. 6A & 6Bis presented by way of example only. The size and configuration of thefield-replaceable unit of the multifunction coolant manifold structuremay depend, in part, on the available size within the associatedelectronics assembly or electronics rack to which the cooling systemprovides cooling.

Note that the coolant-commoning manifold 320′ cross-section is madelarger than normally required in order to allow the returning coolant toslow down, allowing air and other gas in the coolant to de-aerate, orcome out of solution, within the coolant-commoning manifold, with thegas bubbles rising to the auxiliary coolant reservoir portion of themultifunction coolant manifold structure via the conduit 602, whilecoolant from the reservoir replaces the gas bubbles within thecoolant-commoning manifold. Advantageously, removal of air or other gasenhances effectiveness of the coolant since cooling of the electronicsystems is related to the mass flow rate of the coolant.

As noted, the multifunction coolant manifold structure may furtherincorporate, for example, in association with the field-replaceableunit, one or more level sensors 324′ to allow the cooling systemcontroller to know the current coolant level state, and take or signalfor action, if required. Additional features or connections may beprovided in the field-replaceable unit to facilitate installing vacuumbreakers 325′ and/or pressure-relief devices 326′. Alternatively, apressure-relief valve 611 may be provided as described above inconnection with FIGS. 6A & 6B. These devices and valves are placed, inone embodiment, at the highest coolant location within the coolingsystem. This ensures that, even if the device or valve should fail in anopen state, no coolant will escape since the coolant is under little orno pressure within the multifunction coolant manifold structure. Duringnormal operation, the device or valve can fail in place, and not causeany functional problems within the cooling system. The component(s) canalso be safely removed in association with the field-replaceable unit,while the cooling system is operational.

Mounting bracket 620 may be provided in association with multifunctioncoolant manifold structure 400′ for convenient mounting of themultifunction coolant manifold structure within a frame or rack. Fillingand draining of the cooling system is facilitated by providing one ormore fill or drain ports 328′ in association with the multifunctioncoolant manifold structure. In the embodiment of FIGS. 6A & 6B, a fillor drain port 328′ is provided in association with the field-replaceableunit 600. This, along with discharge hose 521, may be used to fill thecooling system or drain the cooling system, in a manner as describedabove in connection with the embodiment of FIGS. 5A & 5B.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprise” (andany form of comprise, such as “comprises” and “comprising”), “have” (andany form of have, such as “has” and “having”), “include” (and any formof include, such as “includes” and “including”), and “contain” (and anyform contain, such as “contains” and “containing”) are open-endedlinking verbs. As a result, a method or device that “comprises”, “has”,“includes” or “contains” one or more steps or elements possesses thoseone or more steps or elements, but is not limited to possessing onlythose one or more steps or elements. Likewise, a step of a method or anelement of a device that “comprises”, “has”, “includes” or “contains”one or more features possesses those one or more features, but is notlimited to possessing only those one or more features. Furthermore, adevice or structure that is configured in a certain way is configured inat least that way, but may also be configured in ways that are notlisted.

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements in the claims below, if any, areintended to include any structure, material, or act for performing thefunction in combination with other claimed elements as specificallyclaimed. The description of the present invention has been presented forpurposes of illustration and description, but is not intended to beexhaustive or limited to the invention in the form disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the invention.The embodiment was chosen and described in order to best explain theprinciples of one or more aspects of the invention and the practicalapplication, and to enable others of ordinary skill in the art tounderstand one or more aspects of the invention for various embodimentswith various modifications as are suited to the particular usecontemplated.

What is claimed is:
 1. A method comprising: providing a cooling system,the providing of the cooling system including: providing a coolantsupply manifold; providing a multifunction coolant manifold structure,including: a coolant-commoning manifold; an auxiliary coolant reservoirdisposed above and in fluid communication with the coolant-commoningmanifold; and providing multiple cooling structures coupled in parallelfluid communication between the coolant supply manifold and thecoolant-commoning manifold to receive coolant from the coolant supplymanifold, and exhaust coolant to the coolant-commoning manifold; andwherein the coolant-commoning manifold is sized to slow therein a flowrate of coolant exhausting from the multiple cooling structures to allowgas within the exhausting coolant to escape the coolant within thecoolant-commoning manifold, the escaping gas within thecoolant-commoning manifold rising to the auxiliary coolant reservoir,and being replaced within the coolant-commoning manifold by coolant fromthe auxiliary coolant reservoir.
 2. The method of claim 1, wherein themultifunction coolant manifold structure is a single, integrated andrigid structure, and wherein the coolant-commoning manifold has a largerdimension in a first direction compared with that of the auxiliarycoolant reservoir, and the auxiliary coolant reservoir has a largerdimension in a second direction compared with that of thecoolant-commoning manifold.
 3. The method of claim 2, wherein the firstdirection is a vertical direction, and the second direction is ahorizontal direction.
 4. The method of claim 1, wherein the auxiliarycoolant reservoir is coupled in fluid communication with thecoolant-commoning manifold via a detachable coolant conduit.
 5. Themethod of claim 4, wherein the multifunction coolant manifold structurecomprises a field-replaceable unit, the field-replaceable unitcomprising the auxiliary coolant reservoir, and including one or morecomponents for at least one of monitoring or controlling one or morecharacteristics of the coolant within the multifunction coolant manifoldstructure.
 6. The method of claim 5, wherein the multifunction coolantmanifold structure is configured for the field-replaceable unit to bereplaceable while the cooling system is operational, with coolantexhausting from the multiple cooling structures to the coolant-commoningmanifold of the multifunction coolant manifold structure.
 7. The methodof claim 1, further comprising providing a coolant pumping assemblycoupled to circulate coolant through the cooling system, themultifunction coolant manifold structure comprising a coolantdistribution manifold portion with multiple coolant distributionconnections therein for coupling the coolant pumping assembly in fluidcommunication therewith, wherein the multifunction coolant manifoldstructure comprises both the coolant-commoning manifold and the coolantdistribution manifold portion integrated together in a single, rigidstructure, and wherein the coolant-commoning manifold is sized largerthan the coolant supply manifold.
 8. The method of claim 7, wherein thecoolant pumping assembly comprises at least two modular pumping unitscoupled in parallel fluid communication to the multifunction coolantmanifold structure for pumping coolant in parallel from themultifunction coolant manifold structure.
 9. The method of claim 1,further comprising providing a coolant fill port within the auxiliarycoolant reservoir, below a top of the auxiliary coolant reservoir toassist in defining a gas pocket in an upper portion thereof as a coolantexpansion region.
 10. The method of claim 1, wherein the auxiliarycoolant reservoir further comprises a vacuum breaker to preventcavitation within the cooling system, and a pressure-relief valve toprevent the cooling system from over-pressurizing.
 11. The method ofclaim 1, further comprising providing one or more coolant level sensorsassociated with the auxiliary coolant reservoir to sense level ofcoolant within the multifunction coolant manifold structure.